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Researchers at NIET and elsewhere have studied the effectiveness of TAP: The System for Teacher and 
Student Advancement (TAP) in raising student achievement, improving the quality of instruction, and 
increasing the ability of high-need schools to recruit, retain and support effective teachers. This document 
describes some of the most recent results that have emerged from the research on the TAP System to 
date. Data collection and analysis efforts are ongoing, and the findings described here will be updated 
periodically as information becomes available. 
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NIET Mission 


Recognizing that an effective teacher is the most important school-based factor impacting student 
achievement (Darling-Hammond, 2006; Goldhaber, 2002; Hanushek, 2013; Headden, 2014; Rivkin, 
Hanushek, & Kain, 2005; Simon & Johnson, 2013; TNTP, 2012), NIET is committed to ensuring a highly 
skilled, strongly motivated, and competitively compensated teacher for every classroom in America. NIET 
supports states, districts and schools in recruiting, developing, supporting, and retaining high-quality 
human capital in order to raise achievement levels for all students (National Institute for Excellence in 
Teaching, 2015). NIET provides both on-site and online support across multiple aspects of educator 
effectiveness through educator evaluation, professional development and teacher leadership. This 
support is delivered both on-site and online through the comprehensive TAP™: The System for Teacher 
and Student Advancement (TAP) and customizable Educator Effectiveness Best Practices. As of the 2016- 
17 school year, NIET initiatives are impacting over 200,000 educators and more than 2.5 million students. 


TAP: The System for Teacher and Student Advancement Description 

Introduced in 1999, “the TAP System has grown significantly as a comprehensive educator evaluation and 
support model for increasing educator effectiveness” (National Institute for Excellence in Teaching, 2015, 
p. 4). The TAP System creates multiple career paths for teachers, provides ongoing applied professional 
development using a rigorous rubric of evaluation, and provides performance-based compensation to 
teachers and administrators. Each of these core elements is discussed below. For more information, visit 
www.niet.org. 


e Multiple career paths. In TAP schools, skilled teachers have the opportunity to serve as master and 
mentor teachers, receiving additional compensation for providing high levels of support to career 
teachers and increasing instructional effectiveness across the faculty. Master and mentor teachers 
form a leadership team, along with administrators, to deliver school-based professional support and 
conduct evaluations with a high level of expertise. 


e Ongoing applied professional growth. Led by master and mentor teachers, TAP teachers participate 
in weekly meetings where they examine student data, engage in collaborative planning, and learn 
instructional strategies that have been field-tested in their own schools. Teachers benefit from a 
national TAP database of instructional strategies and their colleagues' experiences. Professional 
development continues in the classroom as master teachers model lessons, observe classroom 
instruction, and support teachers’ pedagogical improvement. 


e =Instructionally focused accountability. TAP teachers are observed in classroom instruction several 
times a year by multiple trained observers, including principals and master and mentor teachers, using 
rubrics for several dimensions of instructional effectiveness. Evaluators are trained and certified, and 
leadership teams monitor the reliability and consistency of evaluations in their schools. These 
classroom evaluations are complemented by analyzing student achievement growth, rounding out a 
multi-measure system of teacher evaluation. Evaluation results are used as formative feedback in 
one-on-one mentoring sessions, and guide planning for individualized professional development. 


e Performance-based compensation. TAP teachers have the opportunity to earn annual bonuses based 
on their observed skills, knowledge, and responsibilities, their students’ average achievement growth, 
and school-wide achievement growth. Master and mentor teachers receive additional compensation 
based on their added roles and responsibilities, and principals can earn additional compensation 
based on school-wide achievement growth and other measures of effectiveness. 
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Educator Effectiveness Best Practices Center Description 

The NIET Educator Effectiveness Best Practices Center provides innovative services, support, and solutions 
to schools, districts, and states to improve educator effectiveness. Based on more than a decade of 
experience in schools across the country, NIET works with its partners to redesign educator evaluation, 
deliver effective professional development, implement performance-based compensation systems, and 
train teacher leaders in schools. The BPC offers a network of expert trainers and access to a range of 
innovative Web-based resources and tools. For more information, visit www.niet.org/best- practices- 
center/services/. 


NIET Initiative Comparisons and Results 


Since 1999, the TAP System of comprehensive educator effectiveness has operated across multiple states 
in hundreds of schools. TAP has grown steadily in the number of schools participating, the majority of 
which are high-needs schools (over 90% of participating schools). In addition to schools and districts 
implementing the full TAP System, a number of states, districts, and schools are using TAP System core 
elements — in particular, the online portal which houses hundreds of hours of effective teaching across 
subjects and grade levels, the TAP teacher observational rubric, and the certification and recertification 
process for educator evaluators. 


TAP System Evaluation Methods Compared to Traditional Evaluation Methods 

To improve the quality of classroom instruction, the quality of each teacher’s instruction must be 
assessed. Traditional school systems have not been successful at measuring and assessing classroom 
instruction. The New Teacher Project (TNTP) published a revealing report in 2009 showing that schools 
fail to evaluate their teachers in any meaningful way (Weisberg, Sexton, Mulhern, & Keeling, 2009). As 
TNTP reported, most teachers were rated at the very highest levels (replicated as Figure 1 below), despite 
the fact that most schools were not performing at these highest levels on achievement indicators. 


A 2014 report from the National Council on Teacher Quality (NCTQ, 2014) indicated that while 
improvements in educator evaluation have occurred since the 2009 report, still many of the same 
problems persist. Given that differences in teacher effectiveness represent the single most important 
school-related factor affecting student learning, accurately measuring differences in teacher performance 
is critical to the improvement of teaching and learning. 
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Teacher Evaluation Rating 
Figure 1. Traditional Teacher Evaluation Scores.* 


1 Teacher evaluation in five urban school districts, based on data taken from 
http://widgeteffect.org/downloads/TheWidgetEffect.pdf. Scores on a 3-point and 4-point scale have been interpolated 
to a 5-point scale using a cumulative probability density function based on reported data. 
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In contrast to traditional evaluation methods noted previously in Figure 1, the TAP System has developed 
a comprehensive approach to teacher evaluation and incentives that depends on multiple measures of 
both teaching practice and teaching outcomes. This system provides differentiated feedback for teacher 
improvement, in contrast to the often found inflated ratings found in evaluation systems (see Figure 2). 
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Note: Teacher Skills, Knowledge, and Responsibilities distribution of TAP evaluations using 1-5 scale in half-point 
increments. Figure based on 10,281 teachers and approximately 41,124 observations, 2014-15 


Figure 2. Observational Ratings of Teachers in TAP Schools. 


The above ratings are based only on the classroom evaluation component of the TAP System, before 
considering student learning growth measures. Teachers are observed several times a year by multiple 
trained and certified raters who consider 19 areas of effective instructional practice. These observers use 
a multi-dimensional, research-based set of standards and rubrics that are fair, transparent, and 
curriculum-independent. Results are provided immediately as feedback to the teacher in post- 
observation mentoring sessions. The scores from all observations of these 19 classroom indicators are 
combined with seven responsibility indicators at the end of the school year to create an overall Skills, 
Knowledge, and Responsibilities (SKR) score for each teacher. On a scale of 1.0 to 5.0, 1 represents 
unsatisfactory performance on a certain standard, 3 represents proficiency on that standard, and 5 
represents exemplary performance on that standard. Teachers earn scores in increments of 0.5. 


TAP Teachers Demonstrate Consistent Improvement across Time 

TAP results show a steady improvement in observed skills during the course of the school year. Figure 3 
shows recent improvement for teachers in TAP nationally. This shows the improvement in instructional 
quality scores over a two-year period. In the data shown, despite a slight dip over the summer, teachers 
demonstrated an overall path of improvement that continued over both years. This graph is based ona 
sample including all TAP schools during the years 2012-2014. 
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We tracked a cohort of 5,469 teachers through observations grouped into six periods in fall, winter and 
spring of the 2012-2013 and 2013-2014 school years. The cohort was composed of teachers working in 
TAP schools both years, with observations in each of the observation periods. Teachers present in only 
one school year or who lacked an observation in one of the quarters were excluded from the sample. 
Including teachers present in only some of the periods would have complicated efforts to compare the 
level of instructional quality at different time points, since each average could reflect substantially 
different groups of teachers. 
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Figure 3. Teacher Instructional Improvement across Time. 


Within the TAP System, improvements to teacher instructional practices translate into gains in student 
achievement. More importantly, these improvements increase over time. 
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Note: Figure created with raw school-level, composite math and reading score data provided by SAS® for control 
(n=3,870) and TAP schools (n=353). 


Figure 4. Student Achievement Growth by TAP System Status. 
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TAP Impact in Schools and Districts 


A notable success of the TAP System is the expanding preponderance of performance data that comes 
from examining the impact of the system across multiple locations and conducted by different researchers 
using varied methodological frameworks (Algiers Charter School Association, 2011; Barnett, Hudgens, & 
Alexander, 2016a; Barnett, Rinthapol, & Alexander, 2015; Barnett, Rinthapol, & Hudgens, 2014; Barnett, 
Wills, Hudgens, & Alexander, 2015; Buck & Coffelt, 2013; Daley & Kim, 2010; Hudson, 2010; Schacter & 
Thum, 2005; Schacter et al., 2002; Schacter, Thum, Reifsneider, & Schiff, 2004; Solmon, White, Cohen, & 
Woo, 2007). The next section discusses several of these studies demonstrating the impact of the TAP 
System. Following this general review, two new studies, a project spotlight, and results from the 2016 TAP 
Attitude Survey are presented. 


Results from Prior Selected Studies of TAP’s Impact 


Mann, Leutscher, and Reardon (2013) examined the impact of the TAP System across fifteen schools 
in Louisiana. In order to determine impact, a one-to-one nearest-neighbor matching algorithm with 
replacements was created to find a comparison school for each TAP school. Based on the propensity 
scores computed using the selection model, the algorithm chooses the non-TAP school with the 
propensity score closest to the propensity score of the TAP school. There was no significant difference 
between the TAP schools and their matched comparison schools in the pretreatment year, t(26) = 
0.080, p > 0.05. However, results of the study indicate that in the four primary subjects assessed, there 
was a significant effect in favor of the TAP schools for ELA: F(1, 6421) = 6.334, p = 0.012; Mathematics: 
F(1, 6421) = 86.386, p = 0.000; Science: F(1, 7084) = 31.792, p = 0.000; and Social Studies: F(1, 7085) 
= 87.411, p = 0.000]. Further, the study examined the impact of the TAP System across time to find 
that the TAP schools significantly outperform comparison schools, F(1, 24) = 5.30, p = 0.031. The study 
also found that 92% of teachers reported that TAP made a positive difference on student achievement 
in their school and 91% reported that the AYP status was improved as a result of the TAP System. 


In 2010, Hudson examined the effect of the TAP System on student achievement across 151 schools 
in 10 states. Hudson used a statistical control matching method to ensure that the TAP schools and 
the comparison schools were equivalent prior to the intervention being implemented. Hudson also 
used a differences-in-differences approach to further account for any differences between the groups 
and to ensure that the evaluation was able to isolate the impact of the program. Results of the study 
indicate that students in TAP schools outperformed students in comparison schools by approximately 
0.15 standard deviations in mathematics, and smaller effects but in favor of the TAP schools in 
reading. Hudson explains these findings in context to other education interventions by noting that 
“the estimated effect of TAP on mathematics achievement is more than twice as large [as class size 
reduction effects]” (p. 28). 


In 2007, Solmon et al. analyzed the impacts of the TAP System in terms of value-added gain scores 
across 650 classrooms in six states, including 61 TAP schools and 285 control schools. Researchers 
analyzed the student achievement gains at two levels of comparison—teacher-to- teacher and school- 
to-school. To evaluate TAP teachers (and similarly in evaluating TAP schools), researchers calculated 
the effect of each teacher on student progress as assessed by the difference between the actual 
average scores of the teacher’s students and the expected average scores of those students (as 
derived from previous scores). Through this process, researchers created a statistical control group 
for the TAP teachers based on performance. Results of the study indicate that in every state more TAP 
teachers demonstrated statistically significant at or above average amount of student achievement 
growth than control group teachers. Further, TAP schools outperformed their controls in 57% of the 
categories in math and in 67% of the categories in reading. 
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e In their 2002 study, Schacter et al. analyzed the growth in achievement of students (n=3,319) whose 
schools implemented the TAP System compared to the growth of achievement of students (n=7,055) 
from matched comparison schools. The schools were matched on achievement (percentile rank in 
Reading, Mathematics, and Language), school size, percent of students eligible for free lunch, school 
configuration, and location. A statewide cluster analysis was conducted to match the schools. Beyond 
the matched comparisons, the results in achievement were based on a multi-level value-added model 
utilizing prior test scores as covariates. Results of the analysis revealed that TAP schools made 
significantly higher improvements in student achievement gains. Further, this study found that those 
schools that implemented the TAP System with higher fidelity more significantly outperformed 
comparison schools. 


e In their 2004 follow-up study, Schacter et al. examined the impact of the TAP System across 11 
schools. The same cluster level analysis with multi-level multivariate analyses were employed using 
all available covariates to compare growth between the TAP and control schools. Results from the 
study indicate that 65% of the TAP schools outperformed their matched controls in reading, language, 
and mathematics achievement, with the magnitude of change ranging from 6% to 46%. The teacher 
satisfaction component of this study indicated strong support for the four core principles of the TAP 
System. 


e In 2014, Barnett, Wills, and Kirby evaluated the impact of the TAP System across 66 schools in 
Louisiana using two rigorous analytic strategies. First, the authors employed a linear regression to 
compare 2012-13 Assessment Index (Al) performance of the 66 TAP schools and non-TAP schools 
statewide, controlling for prior (2010-11) achievement, percentage of students receiving free or 
reduced-price lunch, school configuration, school size (number of students), and percentage of English 
language learners. Controlling for the covariates, implementation of the TAP System showed a 
significant positive effect on 2012-13 achievement: the 66 TAP schools scored 3.7 points higher on 
average than non-TAP schools (p < .01). Second, the authors compared the 66 TAP schools with a 
propensity score matched group of non-TAP schools. The average 2012-13 Al for TAP schools (64.45) 
was 5.47 points greater than the average for the matched controls (58.98). 


e In their 2014 study, Barnett and Hudgens drew upon a sample of 12,095 teacher-level records, 
representing 413 schools in 10 states, to examine teacher retention rates in schools that implemented 
the TAP System during the 2010-11, 2011-12, and 2012-13 school years. Counter to national trends, 
the authors found that TAP System schools retain 14% more teachers than similar high-needs 
comparison schools and 10% more than the national average. Furthermore, the results indicate 
teachers retained in TAP schools become more effective over time as measured by observational and 
value-added scores. 


e In 2016, Barnett, Wills, & Hudgens examined the impact of TAP implementation on teacher 
effectiveness and student achievement in Orangeburg Consolidated School District 3 (OGB3). Drawing 
from teacher observation scores the authors found that in 2014-15, 95% of teachers met the 
effectiveness criterion, an 8% increase from 2013-14 and an 11% increase from 2012-13. The authors 
also compared the student achievement of OGB3 TAP schools to: 1) other schools with similar student 
profiles, using the comparison information provided by the South Carolina Department of Education 
report cards, and 2) a comparison group of covariate matched schools. Results indicate that from 
2011-12 to 2014-15, OGB3 elementary, middle, and high school students improved in the percentage 
of students meeting or exceeding standards in math and science, relative to comparison schools. 


e Barnett and Wills (2016a) examined the impact of the TAP System on Black-White and Hispanic-White 
achievement gaps in Indiana schools. First, the authors compared achievement gaps in 32 TAP schools 
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that began implementing TAP in school year 2010-11 with other Indiana schools. Study results indicate 
TAP schools have narrowed the Black-White gap in ELA and Hispanic-White gap in ELA and math more 
than other Indiana schools have. Second, the authors selected propensity score matched control 
schools separately for 1) the 20 TAP schools reporting data for Black and White students in 2009-10 
and 2013-14, 2) the 24 TAP schools reporting data for Hispanic and White students in 2009-10 and 
2013-14. Study results indicate TAP schools outperformed their matched schools; while achievement 
gaps at the TAP schools narrowed, most gaps at the matched schools widened. 


e Barnett and Wills (2016b) investigated the stability over time of classroom observation scores from 
the TAP System rubric, as well as the relationships between TAP observational scores and value-added 
measures. The correlations between average SKR scores in the 2010-11, 2011-12, and 2012-13 school 
years were highly significant and large, particularly for consecutive school years. Further, the 
examination of correlations between average SKR scores and classroom value-added scores in the 
2010-11, 2011-12, and 2012-13 school years revealed highly significant correlations. To put these 
findings into context, the authors examined the reported correlations across other observational 
measures used in large-scale research studies and discovered these correlations exceed or match the 
range reported in recent literature — evidence that the TAP observation rubric measures aspects of 
teacher practice that contribute to improved student test performance. 


Results from New Selected Studies of TAP’s Impact 


The TAP System in Rural Tennessee 

With support from a federal Teacher Incentive Fund (TIF-4) grant, NIET partnered with five rural 
Tennessee school districts to implement TAP in eleven schools beginning in the 2012-13 school year. The 
first year (2012-13) was a planning year; implementation of the TAP System began during the 2013-14 
school year. Barnett and Hudgens (2016) examined the impact of TAP implementation on teacher and 
administrator effectiveness and student achievement. 


The authors drew from Skills, Knowledge, and Responsibility (SKR) scores as a measure of teacher 
effectiveness. To be considered effective, teachers must have received an SKR score of 2.5 or higher. In 
both 2013-14 and 2014-15, 100% of teachers met the SKR effectiveness criteria. As shown in Figure 5, 
examination of the overall SKR mean indicates improvement in pedagogical practices over time; the 
overall SKR mean increased from 2013-14 (M = 3.56, SD = 0.40) to 2014-15 (M = 3.76, SD= 0.44). Figure 5 
also shows that teachers who were retained from 2013-14 to 2014-15 and, therefore, had SKR scores at 
both time points, made significant improvements in instructional skills from 2013-14 (M = 3.56; SD = 0.40) 
to 2014-15 (M = 3.81; SD = 0.41), t(272) = 12.76, p = .00, d= 0.61. 
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Figure 5. TN TIF-4 Mean SKR Scores, by Year. 
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The authors drew from school-wide student growth scores, performance on the Tennessee Instructional 
Leadership Standards (TILS), and scores on the TAP Leadership Team (TLT) Observation Rubric as measures 
of administrator effectiveness. From 2013-14 to 2014-15 there was a 6% increase in the percentage of 
principals achieving at least one year’s expected growth of students in their schools. Additionally, in 2014- 
15, 94% of administrators obtained TILS scores of at least 3, a 3% increase from the percentage meeting 
this criteria in 2013-14. Further, as shown in Figure 6 below, the average TLT score increased .50 from 
2013-14 to 2014-15. Figure 6 also shows that administrators who were retained from 2013-14 to 2014- 
15, and, therefore, had TLT scores at both time points, made significant improvements from 2013-14 
(M = 2.94; SD = 0.54) to 2014-15 (M = 3.44; SD = 0.54), t(15) = 3.65, p = .002, d = 0.92. 
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Figure 6. TN TIF-4 Mean TLT Scores, by Year. 


As teacher and administrator effectiveness improved, the authors examined the impact on student 
achievement. Examining value-added scores across time, the authors found ten of the eleven schools 
remained stable or improved. Further, across 2013-14 to 2014-15, the overall school-wide value-added 
mean increased over time, from 2.55 (SD = 1.86) in 2013-14 to 3.09 (SD = 1.87) in 2014-15, a significant 
increase (see Figure 7). 
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Figure 7. TN TIF-4 Mean School-wide Value-Added Score, by Year 
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TAP System Perspectives: Administrator Perceptions of the TAP System Across Time 
Barnett, Hudgens, and Alexander (2016b) longitudinally explored administrators’ perceptions of the TAP 
System and its impact. The authors drew from the TAP Attitude Survey data of 167 administrators, 
representing 119 schools in 11 states, who worked in TAP schools in both the 2013-14 and 2014-15 school 
years. Study results revealed administrators report favorable opinions of the elements of TAP. 
Additionally, administrators perceived positive TAP System impacts, and perceptions of these impacts 
have strengthened over the course of TAP System implementation. 


The authors found that support for the TAP System is high and growing among administrators. Figure 8 
demonstrates administrators’ strong level of support for the four core elements of TAP?, and further 
shows the sustained high level of support over time. Administrators consistently expressed the strongest 
support for the instructionally focused accountability component. At both examined time points, 
approximately 99% of administrators reported that that they moderately or strongly supported this multi- 
measure system of teacher evaluation. As one administrator wrote, “TAP has provided a researched- 
based evaluation tool that helps teachers with professional development and strategies that they can use 
in their class immediately.” 
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Figure 8. Level of Administrator Reported Support for TAP System Core Elements. 


Critics of performance measures and incentives for teachers often indicate such policies will result in 
competitiveness and a loss of collegiality among teachers. Notwithstanding, the authors find evidence of 
a high degree of collegiality in TAP schools. In both 2013-14 and 2014-15, 100% of administrators reported 
that they moderately or strongly agreed with statements reporting a high level of collegiality in their 
schools. Further, as shown in Figure 9, the percentage of administrators reporting strong agreement 
increased over time, such that by 2014-15, 97% of administrators reported strong agreement. 


? The five dimensions represented in this and the next figure are constructed from multiple administrator survey items 
using factor analysis. Most items in the survey are based on a 5-point Likert scale indicating agreement (1=Not at all and 
5=Very Much). For reporting purposes on the four TAP elements and collegiality, the results are presented as Moderate 
(weighted average of 3 on the items for that factor) and Strong (weighted average of 4 or 5 on the items for that factor). 
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Figure 9. Level of Administrator Reported Support for Collegiality. 


In addition to reporting support for the core elements of the TAP System, administrators reported multiple 
impacts of TAP. Administrators reported that implementation of the TAP System has impacted the 
effectiveness of their teachers. Administrator agreement with the statement that TAP has made their 
teachers more effective overall significantly increased from 2013-14 (M = 4.08, SD = 0.86) to 2014-15 
(M = 4.23, SD = 0.72), t(159) = 2.19, p = .03, d= 0.20. One principal explained, 


TAP has had a tremendous impact on our school. Teachers feel more confident in the classroom 
as a result of the professional development and understanding of the rubric. | have asked several 
teachers to compare their teaching now with how they taught prior to TAP. All of the teachers 
explained how their teaching is much more effective now than in previous years. None will go 
back to teaching the ‘old way’. 


Further, administrators reported that the TAP System impacted teacher retention. The authors created a 
“retention” construct using five survey items. Over the course of TAP implementation, administrators 
report that their ability to recruit and retain teachers has significantly increased (2013-14: M = 3.23, 
SD = 0.96; 2014-15: M = 3.39, SD = 0.97, t(164) = 2.53, p = .01, d= 0.16). One administrator wrote, 


TAP has had a profound impact on the culture shift occurring at my school. As ineffective teachers 
that have for many years been able to hide behind an inadequate system of evaluation are 
exposed, they have either seen the need for growth or have left the school. As each of them 
leaves, their replacements are better qualified. 


Project Spotlight: Arizona Ready-for-Rigor 

With support from a U.S Department of Education Teacher Inventive Fund (TIF-3) grant, the Mary Lou 
Fulton Teachers College at Arizona State University (ASU) partnered with the National Institute for 
Excellence in Teaching, 10 Arizona public school districts, and the Arizona Department of Education to 
implement the TAP System in 58 schools. Titled the Arizona Ready-for-Rigor (AZRfR) Project, the grant 
provided the participating schools with continuous assistance and support from master and mentor 
teachers, professional development opportunities, and competitive payouts based on effectiveness 
ratings generated through classroom observations and growth in student achievement. Schools and 
districts joined the project at different phases of the five-year grant. The ASU College Research & 
Evaluation Services Team’s evaluation of its impact revealed successes in the following five areas: 


1. Increases in Teacher Effectiveness: Teacher effectiveness was based on SKR scores, school-wide value- 
added, and classroom value-added (if applicable). The percentage of teachers deemed effective 
increased from 62% in 2011-12 to 74% in 2014-15 (Figure 10). 

2. Increases in Administrator Effectiveness: Administrator effectiveness was based on four measures for 
principals and three measures for assistant principals. The percentage of administrators deemed 
effective increased from 31% in 2011-12 to 67% in 2014-15 (Figure 10). 
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Figure 10. Arizona Ready-for-Rigor Teacher and Administrator Effectiveness. 


3. Increases in Retention of Highly Effective Educators: The percentage of teachers who met 
effectiveness criteria in one school year and were retained into the next school year was examined. 
As shown in Figure 11, the percentage of effective teachers who were retained increased each year. 
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Figure 11. Arizona Ready-for-Rigor Retention of Highly Effective Teachers. 


4. Improvements in School Culture: Teachers responded to a survey asking for opinions on aspects of 
working conditions, including time, facilities and resources, empowerment, and decision making. The 
percentage of teachers responding favorably on elements of school culture increased from 46% in 
2011-12 to 78% in 2014-15. 


5. Improvement in Student Achievement: Arizona schools received A-F Letter grades as part of the 
accountability grading system for all public charter and district schools. For schools that began 
implementing the TAP System in 2010-11, the number of schools earning an A increased from 0 (of 9) 
in 2010-11 to 3 (of 9) in 2014-15. 
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National TAP Attitude Survey — 2016 


Teacher Results — 2016 TAP Attitude Survey 

We find evidence of a high degree of collegiality in TAP schools. In the 2016 TAP national survey of teacher 
attitudes, 94% of teachers in TAP schools agree with statements reporting a high level of collegiality in 
their schools, and over 70% report strong agreement. This evidence for collegiality has been remarkably 
high over the last decade, as shown in Figure 123, which indicates that TAP System schools consistently 
have a collaborative and collegial environment. 


Collegiality 
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Figure 12. Level of Reported Support for Collegiality from Teachers Nationwide. 


Beyond the overall high levels of reported collegiality within TAP schools, the levels of support for the 
specific elements of TAP including multiple career paths, ongoing professional growth, instructionally 
focused accountability and performance-based compensation are also high and growing, as shown in 
Figures 13-16. 


A growing number of teachers report that the creation of teacher leader roles in their school has a positive 
impact on student achievement and school goals. Teacher leaders and administrators form a leadership 
team that articulates school goals and supports each teacher in developing and achieving their own 
instructional goals based on their skills and their students’ needs. Teacher leadership roles also provide a 
pathway for teachers to make a greater contribution to the instructional excellence of a school without 
leaving the classroom. 


Multiple Career Paths 
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Figure 13. Level of Reported Support for Multiple Career Paths from Teachers Nationwide. 


3 The five dimensions represented in this and the next four figures are constructed from multiple teacher survey items 
using factor analysis. Most items in the survey are based on a 5-point Likert scale indicating agreement (1=Not at all and 
5=Very Much). For reporting purposes on the four TAP elements and collegiality, the results are presented as Moderate 
(weighted average of 3 on the items for that factor) and Strong (weighted average of 4 or 5 on the items for that factor). 
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In TAP schools, master and mentor teachers lead weekly cluster group meetings where they examine 
student data, engage in collaborative planning, and discuss instructional strategies that have been field- 
tested in their own schools. Teachers benefit from access to a national TAP database of instructional 
strategies and their colleagues' experiences. Professional development continues in the classroom as 
master teachers model lessons, observe classroom instruction, and support teachers’ pedagogical 
improvement. Figure 14 demonstrates the strong level of support for the professional growth that occurs 
in TAP schools, and further shows the sustained high level of support. 


Ongoing Applied Professional Growth 


0% 20% 40% 60% 80% 100% 
m@ Moderate Support ™ Strong Support 
(n=6,416 teachers, 2016 NIET Teacher Survey) 

Figure 14. Level of Reported Support for Ongoing Applied Professional Growth from Teachers Nationwide. 

TAP teachers are observed in classroom instruction multiple times a year by multiple trained observers, 
including principals and master and mentor teachers, using rubrics measuring indicators of instructional 
effectiveness. Evaluators are trained and certified, and leadership teams monitor the reliability and 
consistency of evaluations in their schools. These classroom evaluations are complemented by analyzing 
student achievement growth, rounding out a multi-measure system of teacher evaluation. Evaluation 
results are used as formative feedback in one-on-one mentoring sessions, and guide planning for cluster 
group meetings. Figure 15 illustrates the strong level of support reported by TAP teachers across the 
nation. 


Instructionally Focused Accountability 


0% 20% 40% 60% 80% 100% 
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(n=6,416 teachers, 2016 NIET Teacher Survey) 
Figure 15. Level of Reported Support for Instructionally Focused Accountability from Teachers Nationwide. 
TAP teachers have the opportunity to earn annual bonuses based on their observed skills, knowledge and 
responsibilities, their students’ average achievement growth, and school-wide achievement growth. 


Master and mentor teachers receive additional compensation based on their added roles and 
responsibilities, and principals can earn additional compensation based on school-wide achievement 
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growth and other measures of effectiveness. Figure 16 demonstrates the level of support for the 
performance-based compensation system from TAP teachers. Levels of support for performance-based 
compensation in TAP schools has increased significantly over time, from 49% in 2005 to 79% in 2016. 


Performance-Based Compensation 
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m Moderate Support m Strong Support 
(n=6,416 teachers, 2016 NIET Teacher Survey) 


Figure 16. Level of Reported Support for Performance-Based Compensation from Teachers Nationwide. 


Administrator Results — 2016 TAP Attitude Survey 

The above substantially positive results from the TAP teacher survey are echoed by the 2016 TAP principal 
survey. As shown in Figure 17, principals have overwhelmingly reported that TAP has a positive effect on 
instructional practices, collegiality, student achievement, and teacher effectiveness. Further, as noted on 
Figure 18, a significant majority of principals (88%) agree that TAP helps retain effective teachers. 
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Figure 17. Administrator Survey Results on TAP Outcomes in their Schools 
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Figure 18. “The implementation of TAP has helped retain effective teachers at my school.” 


Recent Research Releases 


Learning on the Job: How Evaluation Systems Can Support Teacher Growth 

Ritter and Barnett (2016) released a paper describing how research-based standards can provide a focus 
for professional development and how effective evaluation can encourage self-reflection and meaningful 
conversations about classroom practice. The authors interviewed approximately 50 teachers, 
administrators, and policy makers engaged in teacher evaluation reform, such as the TAP System, to 
understand how involvement in these reform efforts can change teacher performance. The authors 
conclude that serious implementation of professional, rigorous, and comprehensive teacher evaluation 
systems represents a promising school improvement strategy. Read the full report here. 


A Document Analysis of Teacher Evaluation Systems Specific to Physical Education 

Norris, van der Mars, Kulinna, Kwon, and Amrein-Beardsley (In Editing) conducted an interpretive 
document analysis of three teacher evaluation systems to determine the appropriateness of the 
instruments used in these systems in evaluating physical education teachers. The authors compared these 
instruments to the National Association of Sport and Physical Education (NASPE) tool, which served as the 
basis for the knowledge, skills, and behaviors to assess in physical education settings. Results showed that 
of the three evaluation systems examined, the TAP System had the highest percentage of NASPE tool key 
items, suggesting it is an appropriate tool for evaluating physical education teachers. 
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Upcoming Projects 


Facilitating Dialogues — Voices from Teachers and Administrators 

The annual TAP Teacher and Administrator Attitude Survey has collected responses from approximately 
25,000 teachers over the last decade on teacher attitudes and job satisfaction. The survey contains 
questions that assess the perceptions of the implementation of TAP at the school and teachers’ and 
administrators’ overall job satisfaction. Past summaries have provided a glimpse into those responses 
aggregated across all respondents and comparing early adoption years to more current years. The next 
step in analysis of available data is to examine the changes across time and within each location, as well 
as examine additional questions reported throughout the survey, including the qualitative data and open- 
ended responses. 


Overcoming Barriers in Traditional Higher Education Partnerships 

An innovative collaboration between non-profit, university, and district partners is connecting the 
principles of a proven educator effectiveness model with initial teacher preparation. NIET has partnered 
with the College of Education at Texas Tech University, as well as local partner school districts across Texas 
to connect the pipeline of teacher preparation to NIET’s educator effectiveness model TAP™: The System 
for Teacher and Student Advancement. The authors use transcripts, observations, interviews, and focus 
groups to examine barriers partners faced in this collaboration, as well as the solutions developed to 
overcome these barriers to create a systematic, structured pipeline of teacher evaluation from 
preparation program to in-service teacher. 


Impact of an Educator Effectiveness System on Retaining Administrators Nationwide 

In 2014, Barnett and Hudgens released a report examining teacher retention rates in schools that 
implemented the TAP System. Recognizing the role administrators play in the successful implementation 
of any reform effort, in a follow-up piece, the authors use school rosters, administrator effectiveness data, 
and school data to investigate the retention and impact of administrators who work in TAP System 
schools. 


Conclusion 


The TAP System stands out because of its more than a decade-long track record of growth and success in 
raising student achievement in high-need schools. The research evidence also reveals several key reasons 
for TAP’s positive impact: an evaluation system capable of differentiating teacher performance levels and 
providing detailed feedback for improvement, ongoing professional growth in classroom practice using 
student and teacher data to guide improvement, recruitment and retention of effective teachers, and the 
creation of a challenging, rewarding, and collegial environment focused on high- quality instruction and 
student learning. 
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